Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 99.59%. Comparing base (c650976) to head (557eb1c).
Report is 11 commits behind head on master.

@@             Coverage Diff             @@
##           master    #1034       +/-   ##
===========================================
+ Coverage   86.30%   99.59%   +13.29%     
===========================================
  Files          43       34        -9     
  Lines        2387     2205      -182     
===========================================
+ Hits         2060     2196      +136     
+ Misses        302        6      -296     
+ Partials       25        3       -22     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@AyushSawant18588 thank you for the PR! Let's rename this field to VLLMGuidedChoice to clearly indicate that this is not from OpenAI API. And also let's update the comment accordingly

@AyushSawant18588 thank you for the PR! Let's rename this field to VLLMGuidedChoice to clearly indicate that this is not from OpenAI API. And also let's update the comment accordingly

So like chat_template_kwargs this is an extra_body field, hence its supported by OpenAI as well and is not vllm specific. For example:

from openai import OpenAI
client = OpenAI(
    base_url="http://localhost:8000/v1",
    api_key="-",
)
model = client.models.list().data[0].id

completion = client.chat.completions.create(
    model=model,
    messages=[
        {"role": "user", "content": "Classify this sentiment: vLLM is wonderful!"}
    ],
    extra_body={"guided_choice": ["positive", "negative"]},
)
print(completion.choices[0].message.content)

@sashabaranov wanted to confirm, are the changes made in this PR fine or should I update?

@AyushSawant18588 okay, I see the issue! I've just merged #906, let's introduce ExtraBody here instead

@sashabaranov most of the fields we use are on top level and since this field is widely used for the use case of structured outputs can we merge this PR?
I will raise a separate PR to include extra body in chat completions request

@AyushSawant18588 but it's not part of official OpenAI API right? If it is, could you please point me to the documentation?

@AyushSawant18588 we can make it .NonOpenAIExtensions.GuidedChoice to preserve type-safety

@sashabaranov do these changes look good? made NonOpenAIExtensions.GuidedChoice

@AyushSawant18588 thank you for the update, this seems like a pretty elegant solution!

Here are a few things to consider:

1. Copilot point about ,inline not needed above
2. The NonOpenAIExtensions is a very generic name that could apply to pretty much anything in the codebase, in that context it should be ChatCompletionRequestNonOpenAIExtensions it seems

@sashabaranov Can you check the changes? I renamed struct to ChatCompletionRequestExtensions cause name was getting too big but let me know if it has to be changed